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ABSTRACT ^ . ^ ' 

An analysis system was developed to facilitate the 
comprehensive analysis of over 800 hand^written mathematics, 
communications, and social studies lesson transcripts. The coding 
scheme facilitates the reduction of -qualitative data into categorical 
ds^ta. The heart of the analysis system is the Dominant Instructional 
Event (DIE) which facilitates the* further subdivision of each lesson 
ini:o subcomponents labeled content, materialSif. qroupinqs, interaction 
patterns, student activities, and coqnitive taxonom^c level. Tren^ 
are -reported to illustrate the utility of the analysis system, 
however the paper focuses primarily on tbe fitethodoloqy and the 
potential importance and estimated validity of the analysis system. 
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A SYSTEM FOR CODIFYING HANDWRITTEN LESSON 
o 

TRANSCRIPTS FOR . COMPREHENSIVE ANALYSIS* 

Robert M. Toepper 
, Roberts. Everhart 
Barbara J. Hawley 
James M. McGeever 



I. Our Responsibility 

^ ' During the summer of 1973, we were charged by the director of evalua- 
tion and 3©Winentation of the Experimental Schools Program (ESP) sponsored by the 
% National Institute of Education to attempt to gain a comprehe^sive knowledge of 

the utilization of ESP funds for the purposes of curriculum and instruction in the 
school district we are studying. The .Experimental Schools Program emphasizes 
a comprehensiye change stratcg}', as opposed to one olf piecemeal change, for the^ 

V 

Implementation of educational innovations. The Experimental Schools Prograbti 
granted a great deal of autonomiy to the district regarding how to define the project 
originally and how to execute it over the five years of its operation. Our external 

evaluation program, is responsible for non-participant, summative evaluation 

' ^ ^ > ' i 

(evaluation of the project as a whole, especially as to process and end results) 

while-the ^3istrict*s internal evaluation department is committed to a participatory, 

formative evaluation stance (the use* of evaluation to im'prove a program while it 

is in the process of being operationalized). • 

Our entire evaluation program, a imit fcomposed of nine professional and ' , 

three support staff members, is committed to ^approach called 'illuminative 

*A paper presented in Division C of the American Educational Research 
Association Annual Meeting, Washington, D.C., March 31, 1975. 
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evaluation'' whereby we address the questions: how arc things done in the 



district? and why are they done the way they are done? as well as the question: 
With what impact? The basic methodology employed by our component, the 

r 

Instructional Environment Component, is variously called ethnography, field- 
research, participant or non-participant observation, or qualitative research. 
In general, the methodology is an attempt to employ some of tlie fieldwork 
strategies of anthropological and sociological investigation to educationai insti- 
tutions and phenomena. ^ 



Seie ctic 



IL Sample Selection 

1 

We used a stratified random sampling process, with some variations,-^ 
which resulted in the observation of about 135 teachers. Each teacher was 
observed for four ,to five consecutive lessons^ in the subject area for which that 



One-fourth of the district's elementary teachers, stratified kinder- 
garten through grade 3, grades 4 through 6, and by IGE (Individually Guided 
Education) school units, were i^andomly selected to be observed for both commu- 
nications and mathematics instnaction. One-fourth of the district's secondary ^ 
teachers were randomly chosen by course responsibilities for either communica- 
tions or mathematics streanf observations with the restriction that no secondary 
teacher would be chosen for more than two courses. One-fourth of all* the kinder 
garten through grade 4 social stud^ds classes in the district were chosen pn.a 
random basis, stratified by grade or level. One-half of all the fifth and sixth 
grade teachers were observed with emphasis on those involved withisither of t^vo 
innovative curricula being implemjpi^ted; and one-half of all the junior and senior 
high school instructors were chosen, based on major responsibilities for^gach, 
required social stui^ies CQui^se. - * * ^ 

^Bellack's study of the language of the classroom used four consecutive 
lessons. A. Bellnckr et al. , The Lnngnage of the Classropm , 'Teachers Collci^e 
Press, New York, Columbia tIniversity,^19G6, p. 11. (Continued on next page)) 



teacher was chosen on the sample, a total of about 750 lessons. While consecu^j 
tive observation periods raised some issues with regard to principles of time 
saixipling (such as whether or not four consecutive days in November were neces- 
sarily the same as four in May, or the same as four separate days selected at 
random throughout the school year), ^ we 'felt that the necessity to build and 



(Continued from previous page) Smith and Meux scheduled. five eonsecutive 
lessons. They also chose weeks unbroken by holidays or major schoM events 
and not too close to either end of a marking period, B. O. Smith. and M. Meux, 
et al. , A Study of the Los^ic of Teaching , Urbana, Illinois," University of lUinoig 
Press, 1970, ppf 205 and 207. In a study of verbal classroom interaction, 
Wright and Proctor reported that after six cgnsecutive observations subsequent 
data add little to that already collected. D. M. J, Wrig:ht and V. Proctor, 
Systematic Observation of Verbal Interaction as a Method of Comparing Mathe - 
matics Lessons , Cooperative Research Project No, 816, Office of Education, 
Washington, C, , U,S, Department of Health, -Education and Welfare, June 
1961, 

^Smlth and Meux debated these same issues: 
V The ^estion of how much of a teacher* s classroom dis- 

course to record, and whether to concentrate the recording in 
a brief^ period of time or to distribute it over a month, semes- 
ter, or year, is one which we considered at some length. It 
could be argued that spot recorcKngs over a s^ester or year 
would be more representative of a teacher*s work than an equal 
number of recordings taken consecutively. It would appear 
that spot recordings would tend to cancel out the effects of var- 
iations of content within a course and of changes in style of 
teaching from one topic to another. These are very pogent 
reas®ns. Nevertheless, we decided to make five consecutive 
recordings per teachei:. For one thing, such recordings would 
provide continuity in the teaching of a topic over a period of ' 
days. In this way, we would obtain the^sort of context useful 
in a logical analysis. For another thing, consecutive recording 
is easier to schedule and less disruptive of school routine. 
These are major considerations when the cooperation of a public 
-School is being sought. 
O. 'smith, M. IMeux, et aL , A Study of the Logic of Teaching , Urbana, 
iversW of Illinois Press, 1970, Appendix 1, pp. 205-206. 



mamlain fapporl with staff was puramoimt for the type of information \vc wanted 
to collect, and that continuous observation was a much better vehicle for accom- 
plishingtKis than visiting classes at monthly intervals."^ In retrospect, this 
decision wals wise, especially when placed in juxtaposition to what we were 
originalh/lcd ta believe would bc-the initial ambivalences held by a sizable 
numb^' of teachers about our presence. ^ 

A rather long period of observation^ was considered appropriate in 
order to establish the unobtrusive preserlce of^the observers'' as wel} as to 



Medley and Mitzel recommend that it is helpful if teachers and observ- 
rs mingle socially. D. M, Medley knd H. E. Mitzel, "Measuring Classroom 
hav^or by Systematic Observation, "dn N. LrT Gage (ed.). Handbook of Research 

,1963, p. 



op Teaching, Chicago, Rand McNally, 



307, 



^See also P. W. Jackson, Life In Classrooms , 'New York, Holt, Rinehart 
and Winston, 1968, pp. 131-133, regardihg elementary teachers' qualms over the 
prospects of being observed too frequently, .See* also Medley and Mitzel, p. 247, 

^ . 

> ^ ^Long in comparison to tlie ^'micro-ajjproaches*! to research on class- 

room instruction which tend to study a limited number of dimensions using time 
spans shorter than a lesson as the unit of analysis such as those found in A, Simon 
and E, G. B6yer (eds.). Mirrors for Behavior; An Anthology of Observation 
Instruments (17 vols.), Philadelphia, Pa., Research for Better Schools, Inc., 
1970. Short, however, in cqmparison^*to Smith's one-semester study of Geoffrey's 
classroom, in L. M. Sniith and W. Geoffrey, The Complexities of an Urban Class - 
rQom; An Analysis toward a General Theory of Teaching ^ New York, Holt, 
Rinehart and Winston, 1968. - 

'In a study of classroom interaction, Hudgins and Ahlbrand stopped the 
instruction in the secondary classrooms fte^ were studying four times. in a 50- 
minute class period to ask the students what they were thinking about. After the 
second response on the first dajy, the curve for responses concerning the presence 
of the researcher.s goes down rapidly. B. B. Iludgins and W. P. Ahlbrajid, Jr., 
A ^Study of Classroom Intc;:action , Technical Report Series Number 8, St. Ann, 
Missouri, Central Midwest Regional Educational Laboratory, 1967. 
, ' . An ihtroductt)ry letter to prospective participant secondary schools in 
Smith and Mcux^s study of the logic of teaching; said: (Continued orincxt page)* ^ ^ 
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assure the teachers of the observers' commitment to a comprehensive, natural- 
Istic iinder standing of the teachers' procedures, to give the observers tlie oppor- 
tunity to see^the teachers in a variety of teaching postures, and to minimize the 
possibility that the-teache;rs would ^^put on a show'^ for the observers, * . 

The subject areas selected for observation were: mathematics, commu- 
nications, and social studies, Thte mathematics and communications '^streanis''^ 
were chosen because they were emphasized by the district as beln^ part of a 
comprehensive kindergarten through grade 12 program, especially in the area of 
individualized instruction. Social studies stream instruction was chosen for 
observation because of the planned implementation of several elementary social 
studies programs during the 1973-74 school year, 

' ' ' / ^ ■■•>. 

in, ^ Data Collection Procedures ' 

Three Instmction^l Environment personnel were assigned to monitor 
the classes selected ifi the sample. On the elementary level, two monito3*e^ 



(Continued from previous page) . - , 

To recGord a class session requires two individuals to handle 
. * equipment and to record the non-verbal context of the proceedings. 

The project staff has found that the presence of these individuals 
(graduate students on the project staff) has created no discemibl^e , 
disturbance or disruption of iplassroom activity. After the first ^ - 
ti^y, their preseni:^^ was generally taken for-granted and the 
normal atmosphere of the class reasserted itself,- • ^ ^ * 

*B, O. ^Smith, M, Meux, et al,\ p. liOZ. * . 

In regard to his study of classroom interaction, Bellack states that 
"Alth6?i^ it cannot be assumed that the research procedures had no effect on the 
classroom behavior, it seems reasonable to conclude that this effect was mining^al,'/ 
jBellack, etal., p, 11. - « * . , • 

^The (iistrict's six curricular streams are mathematics, communications, 
social studies, scientific, career education, and personal development, 

^The type of data we collected will be ma(Je clear in section IV, 



ERIC 



o . 7 



shared responsibilities for malhematics and^ommunications instruction and the 
third had responsibility for the observation qS social studies instruction. One 
person was assigned to each stream for the secondary observations. Mathema- 
tics and communications stream observations began on October 17, 1973, and 
ended on June 3 i 1974, while social studies stream observations began on Sep- 

C 

tember 18, 1973, and were completed on June 4, 1974. Monitors generally 
attempted to complete, the sample on a s-chool-by-school basis, finishing the 
sample within each schooLbefore proceeding to the next school. 

Data were gathere.d by taking hanQwritten fieldnotes, recording afe 
many of the,activities as possible of each teacher and class selected in the 
sample. We attempted to impro.ss on teachers that we were not in theiV class- 
rooms t1^ evaluate tl^eir perfqrmance, but rather to describe their classroom 
activities as accurat^y as possible in the subject areas being studied. 12 As 
transcripts were taken, we gave teachers the option of s^eetng those transcripts 
if they so desired. Monitors made an effort to show teachers the transcript for 
the first cjay^s observation and explained to them that they could peruse the rest 



lOSee the previous sectioii on sample selection fbr our rationale for 
continuous observations rather than using some form of time sampliiig. « 

* 

' l^Tape recording of lessons was considered, but deemed inappropriate, 
because of the potential threat to the teachers, the possible disruption of the 
instructional, process, and the recognition that we would coll^ctiar more data 
than we would use. ' . ' ' • ' 

l%edloy and Mltzel, p. 307; Bellack, ct al., p. 11; and Smitli and 
Meux, etal., p. 206. . ^ . ^ 



of the transcripts if they wished. In general, however^ few teachers asked to 
see our subsequent trans'cripts, 13 



IV, Analysis System. 

^. As we approached ttie two-thirds point in our .observations of mathe- ^ 
mat^ics, communications, and social studies stream instruction, we began to 
develop a scheme by which an analysis of our handwritten transci:ipts could be 
coded for later statistical analysis. A review of the research litcratu-^^ gener- 
, ally found extant instructional analysis schemes to lack the 'comprehensiveness 
we desired. Many such analysis schemes, especially the more popular ones such . 
as those developed by Bellack, Flanders, o^ Withall, limit .their foci to class- 
room interaction patterns; otl;ers are oriented toward the observation of specific 
ages or grade levels, unique curricular content, or previously. defineS teaching 
strategies. Therefore we proceeded to develop a model for the analysis of 
instruction to act as an orderly method of, reporting a sizable portion of the data 
in the approximately 750 lesson transcripts that we collected duying the 1973r74 
school year. 

We will now proceed to explain this analysis system in. terms of how 
'^treatments" were defined, the character of the Dominant Instructional Event (DIE), 



■^^^Medley and Mitzcl speculate about the same lack of teacher interest 
in seeing the results of the data collection. Medley and Mitzel, p. 307. 

I'^See B. Rosenshinaand N. Wrst, ''The Use of Direct Observation to 
Study Teaching," inR. M^. W. Travers (cd.). Second Handbook of Research on 
Teach ing , Chicago, Rand McNally, 1973; R. Dreebcn, '^The School as a Work^ 
plaQc,*' la ibid., pp. 4G5-46G; Simon and Boycr, 1970; R.^ T. Ilyman, Teaching ; 
Vnniagce Points for Study , New York, Lippincott, 1968; .or Medley and Mitze-L 



the nature of the six componeftts of each^DIE, and the geiKral manner by which 
each lesson was qoded, 

r 

* 

A. Treatment ^ - ' , 

■ \ . ^ • i 

For purposes of analysis, basic curricular ^'treatments.*' in each stream 

were identified near the completion of the observations. The criteria tliat were 

considered in defining and/or naming curricular treatments included: (1) use of 

the same or similar curricular materials by a given teacher, (2) existence of 

the ^amc or similar curricular materials in several settings, or (3) illustrating 

a relatively consistent approach to the subject-matter by different teachers. 

For example, in social studies, eight such ^'treatments** were identified. 

We will use Exhibit A to help explain how our analysis system works, 

B. Dominant Instructional Events . , ' 

^ • -. . • 

. ' Our Analysis system begins with the codification of Dominant Instruc- 

«■ * * , • 

tional Events (DLE*s). DlE*s are segments of lessons that possess an integrality 

"V . . 

of utilization of content, materials, groupings, interaction patterns, student 

jt 

activities, and cognitive level, preserve the arrangement by .which the dimensions 
of these components occur in time, and are defined by the context of the "natural 
settings" of classroom instinicfion In which they occur, The DIE is designed 
primarily as a heuristic device which facilitates the subdivision of lessons for 
furtlier analysis. 



^.^Sce J. H. Burnett, »' Event Description and Analysis in the Micro- 
ethnography of Urban Classrooms, in F. A. J. lajnni and B. Storey (cds,), Cul- 
tural Relevance and Educa tional Issues , Boston, Little Brown, 1973, pp. 290-291. 
^ 

8 ' 
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Exhibit A 

ANALYSIS X)F SOCIAL STUDIES INSTRUCTION 
Percent of Duration of DIE (min. ) 



Schools 
Grades 
N, Teachers 
Avg. N, Students 
Avg/S/T; 



N, Les^sons: 
; Duration of Lessons (min. ): 
Avg' Duration of Lessons (min.): 



r 

Dohiinant Instructional Event (DIE) 

€ 


New 
Content 


Asgns. 


Pract., 
Act.^ 


Review, 
Testmg 




dumber of DIE 












Duration of DIE (min. ) 












Average Duration of DIE (min ) 












Range, Duration of DIE (min ) • 












DrE (mm. )/Lesson (miii. ) 












Content * ' 


Anthropology 












H/Conomics , • 












Geography 












.ill Story 








r 




Humanities 












Political Science 












I^sychology 












Lfociology 












Materials . 


nU Rfl 1 J OX* nnnk R 












Curriculum Pkgs. 












Tcachcr-Dev. 












Published Suppl. 












Audio-Visual 






r~ 






Groupings 


Whole Class 












Small Groups 












Alone 












Independent 












Interaction patterns 

• 


Teacher Talk 










t 


T. -Initiated/S. 












S. -Initiated/T. 












S./S. 










1 


T. /One S. . 




? — - 








None 












Student Activities 


Correcting 












Test- Taking ^ 


7 ? 










Receiving 


i 


- 








Reading 








— ^ 




Mm. -Dr. ; Smp. Rsp. 












St. Gd., Ex., Quest. 












Writing 












Library Work 












Problem -Solv ing 












Pract. /Art. Croat. 












Discussion 




> 








Games, Sim., Plays 












* 


Cognitive Level 
(percenf^tjf number 
of DIE) 


Knowlf^lge 












Comprehension 












Application 












Analysis 












Synthesis 












Evaluation jf ^ 






1 







Lesson 
Totals 



Four categories compose the DIE possibilities: 

Presentation 'of new content: A segment of a lesson^which 
introduces the st'udont(s) to specific cognitive, affective, 
or psychoTiTiotor content. 

Practice or activities'^ derived froiYi new content presen- 
tations.or.assignmcnts: A lesson seg;nicnt where the 
studentfs) is(are) expected to practice or drill on 
content already presented to them or to participate in 
student activities desij^ned to elaborate or sometimes 
^ , to prepare the student(s) for content presentations. 

Assignments: A lesson segment where the student(s) is 
(are) expected to complete a task or produce a product, 
^ ^ $ . ^ . 

Reviews evaluation, and testing; A lesson segment that 
repeats, measures, or corrects; includes ^^de^briefi^^ 
games, simulations, and role plays,. ^-^ ^ 

We feel that these four possibilities include virtually allthe possible 
tnstrucf^al activities in any mathematics, social studies, or communications 
lesson; Each lesson can be divided into any combination of the four DIE cate- 
gories. That which is not subsumable under these four headingajn eludes such 



16'Tra.ctice*' is more appropriate to DIE^s in th^ mathematics and commu 
nications streams while ''activities" is more applicable social studies lessons. 

J^Our four DIE categories ^.re similar to Flanders' identification of- . 
instrrictional activity periods, whicli are characterized by a major change in 
class formation, the communication pattern, or the subject under discussion. In 
an e:aimplo, Flanders describes five such activity periods: (1) settling down to 
work, which we excluded from our analysis, (2) introducing new material, ^e 
same as our presentation of new content, (3) teacher-directed discussion or 
work on material that is not new, similar tb our practice or activities, (4) super- 
vision and direction of. individual scatwork, comparable to our assignments DIE, 
and (5) periods .of ev^aluation, in which homework and test re&ults are disoissed, 
containing part of what we call ''review, evaluation, and testing." N. A. Flanders, 
Teacher Influence, Pupir A'ttitndcs, and A,chiev(?mcnt , Cooperative Research 
Monograph No. 12, Office Of Education, Washington, D.C., U. S. Department of 
Health, Education and Welfare, 1965, j)p. 19 and 22. 




non-r instructional activities as foil-taking, announcements, or closing^activUies 
which ap^ excluded from analysis in this systeni^ 

C. DIE Components and Dimer^sions ^ 'j,- . - ' ' 

Each DIE is divided into feix major components . The components are: 

content, materials', groupings, interaction patterns, '.student ^tivities, and 
"^cognitive level. We feel that these components' encompass a majority of the 

instructional activities that can occur Avithin any one^ DIE, although other compo- 
^ nents^can probably be added to .our listing. ' ^ * ^ 

Each DIE component is subdivided ^ther into from four to twelve 

dimensions . These dimensions are merely more specific delineal^ions of each 
• DIE component. For ejcample, under the component labeled '^interaction pat- 

terns, »' one can find a variety- of interaction pattex^s possible. The possibilities 

listed—lecture and teacher talk, teacher-initiated/students, student- initiated/ 

teacher, .student/shidon^ teacher/one stucjent at a time, or none—further define 



the interaction pattern component. ^ ^ 

T 

,% 

In summary, then, lesson transcripts are divided first into DIE cate - 
jgories. Ekch categorj^ is further broken down into components consisting of a 
range of defining dimensions . We turn, (low,- to our definitions of components 
and dimensions. ^ ^ 

Content is the first component of each DIE. Each stream has a unique 
listing of content dimensions, developed in order to enable us to categorize all 
the possible subjeU matter that can be taught in that curriculum streain. A 
listing social studies stream^<^ntent dimensions is provided in Exhibits. 

The rest Of the components are the same for all three streams. 
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GHjA 

TKA^^Ht^R } ^ 



OTHKR EL KM: Teacher-developed 
OTflER ELKiM: District i , , , 

CONCEPTS AND_VA_LUES 

^lAN: 'A COURSE OF ^TUDV 
PEpPLE AXl) TECHNOLOGY , 
KSf SUPPLEMENTARY i \ ; 
cTLINlOR IIIC.I! SCHOOL Ji ! ; 

SEkrnR HiGii scHOot 



LESSON NXMBKR 

11' 
NU^lBjERlOF STUDEI^TS 



DURATIO;^^ OF LESSON IjN MI-NUTES 



1 i ' 



PlUv^ENT^\TlON OF NEW CO-NTENT 
PRACTICE OH ACTIVITIES , , [ 
ASSIGNMENTS i i i i 1 I 
REVIEW, TESTING ' ' 



m ELI MtNAW^Wr A 



CIIRRICILUM 



DU?WHe^-OF - DI FKN -?^1fNUTESr- 



ANTHROPOLOGY i , 
ECONOMICS 

GEOGRAPHY ' I 

HISTORY _ ' I 

HUMANITIES ; . ; 
POLITICAL SCIENCE | 

PSYCHOLOGY t i„ 
tofcTOLOGY 



! 1 



DIE 



BA SAL TEXTBOOKS 



CURRICULI'M PACICAGES | -r 
TEACHER-DEVELOPED MATERIAL, 
PUBLISHED SUPPL. MATERIAL ; 

Ahnid-visr \L ' j ' i 



SMjALL GROUPS j i 
ALONE I : 1 I I 

tniSfprkdkxt 



LECTURE-TFACIIER TALK 
TE A CIIERr 1 '1'^ A 1 T':D/ST UDE NX 
^rtJD ENT-I Nl tL\ TED/TEA Oil ER 
STUDENT/STUDENT ^ | ; 
TEACHER/ONE STUDEN^ j [ 
NOKE' ^ ' , I 1 ! 



CORR ECTI XG ' 
TE$T-TAklNG r , 
RECEIVING I 
READING ' J_ 

M EMORI Z A TION-DRILL; 



1 
I 

I i 
€eNTi?NT 



jMATEBL^LS^ 



iGROUPINGS ■ 



•I 



1 



INSTRUCTIONAL STR 

• i . 1 I ' 5 

I I I I f < t t 1 



5 

m 



ION 

-r 
1-1 



lINSTRXJCtridN PATJTERNS 

T- r 



Scl^X-SUIDES 
WRITING 
LII^RARY WORK 
PROBLEM-SOLVIXt] 
PR)\CtlCAL OR ARTISTIC CREATIONS 
DISCUSSION ! . ^ i ' L 



SrM(PLp RESpPONSLS 



GAMES. Snin ATIONS. PLAYS 



KNOWLEDGE , 
COMPREIIKXSION 
AP|PL^CATION ; 
ANAiYSLS 1 



^NTHESLS i 
EVALUATION 



I 



9 



CdGNfTLVE 
L^VE 



TA KONOMIC 



Materials refers to instructional software and hardv^^are that ^an be 
expected to be found in any classroom. The materials component includes the 
following dimensions: ' ' ' . 

Basal textbooks, possiblj^ accompanied by student wor^lc- ' -^^ 
books and tpachers* manif^s ^ ' ' . 

Curriculum packages, a relatively self-contained set of 
instructional materials consisting of audio-vieu^I'^^d/ 
or manipulative as well as textual material^ 

Teacher-developed materials, which can be basic or ' 
supplementary , » ^ * 

Published supplementary materials, including source 
materials as well as study guides or worksheet^ / 

Audio-visual, including pictures, charts, 'maps, globes*, 
transparencies,; phonograph records, tape recordings, ' . 
film strips, films, television programs, videotaped , ' 
programs or activities, teaching machines, and guest 
speakers, but excluding use of chalk boards X)V oyer-- 
^ head projectors - ' ^ , 

> . . ^ 1 , * 

Groupings encompass the basic combinations by whij^h students can be 

organized for instruction. ^The specif f6 dimensions of tiie^oupings component 

in our analysis system are: r ■ * ^ ^ » 

f • ' . , 

\WTiole class / • . - ' " ^ " \- 
' Sn? all groups ; - . \ ' / 

Alone, on an assignment required of me whoje class 
Independent, working alone^at the studfent*s a^,rate on 
, • activities appropriate^ td each ^tuderit^s -own nebds, 

' ' ' ' interests, or abilities * - ' . ^ ^\ 



The interaction patterns coniponent is composed of six dimensions that 

Sttempt to define all the possible inferp.ctiolis that can be found in any classroom; 

AThc?.six*dii1iensions are: ' i * * - ' • 

• / I Eecture and teacher talk' ' ^ ' 

TeacihcrritVitiatcd/stude*k ' \ 

• Student'-initiated/teacheV'. j v 
* "^'^X j^, Student/!?tifdent, includiiig^s'tud6n|; presentations aS well 
ats dislpussion . ' >\ '\ / ' 

^ ' ':'•• !! ' Teachef^oiie student at ^.'time /, ' 



student activities are defined as the general l^esponses expected of 
students in order to faciliUitc their active participation in the learning process. 
Student activities include: . , ' ^ . 

Oorrectingone^s own or other student* s."ivo'rk 

Test-taking , . * ' , 

Receiving audio and/or visual inputs " */ . 

Reading, either orally or silently^ 
Memorization, drill,' or simple response 
^ Con>pleting study guides, exercises, or questions 

Library work, ranging £rom reference skills to.resea.rch 
Writing, including paragraphs, essays, papers; stories, 
or poetry . 

Problem-solnngj including laboratory experiments 
Construction&, hdlidicraft^ practical creations, or artistic 

creations such as drawings, paintings, or sculpjur'es 
Participation in discussion 

"Participation in academic games, sinllilations, role plays, 
or plays 

The final component is cognitive leveL The six Cognitive levels 
include: - / - ' 

knowledge, or memory 
Comprehension 

Application * 

Analysis " ♦ . . 

^ Synthesis 
\ Evaluation 

According to Bloom, and others, in the Taxonomy 6Y Educational Objectives , the 
cognitive domain includes those objectives or outcomes that deal with the recall' 
or recognition of loiowledgc and the development of intellectual abilities ajid 
skills. Although these dimensions are subdivided Into finer elements in Bloom^s 



l^Basod on B. S/ Bloom (ed. )> Taxonomy of Educational O^ectives; The 
Classification of Educational Goals, Handbook I: Cognitive Domain York, ^ 
David McKays Co. , Inc., 1956, , . . 

• ' ^' ,14 

16' 



ERIC 



JC I 



taxonomy (each dimension has up to 12^subdivisions), w#e used only the broad 
dinncnsiQiTS»bccause we were coding what w,c considered to be the actual behavior 
of tfi.e teachers and students we observed, a more gross level of analysis than 
that which could be performed in analysis of the intended behaviors as specified 
by the teachers* objectives. In addition, any such coding involved judgments of 
the instructional process, judgments which, if pushed lo any finer level, would 
result in a specificity which would confoimd our ability to make meaningful 
generalizations, 

. V. Mechanics of Coding 

Seven hundred and forty-seven transcripts were coded following tlie 

analysis scheme we have just described. Each observer analyzed only those 

transcripts that he or she had written. An inter-rater reliability of . 90 was 
achieved among the 'three observers, 19 . ^ . 

Each lesson was analyzed as a separate entity. We first determined 
into which curricular .treatment each lesson, or segment of a lesson, should be 
categ^||:ized. Next, after dividing the lesson into the appropriate number^.of DIE'S, 



nt I 



the '^preliminary data'' shown in Exhibit B were coded. Then, proceeding from 
left to right, the observer placed one (1) in any column where that entry was 
present inlhe lesson. An exception was columns 25 and 26, "duration of DIE in 
minutes," ^where time was recorded instead, - . ^ 



l^See the Appendix ior a discussion of our nieasure of inter-rater reli- 
ability, ' ^ ^ • ^ . 
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Exhibit C represents how one of the 33 Fortran coding forms for the 

analysis of social studies instruction looked upon coinpletion. Proceeding across 

line seven of Exliibit C, we find that at School 8 in graders (coded 11), teacher 

36 taught 30 students for ^|painutes in Man; A Course of Study (column 16). The 

presentation of new content DIE (column 21) lasted 30 minutes (columns 25, 26) 

and covered anthropology (column 27) using a curriculum package (column 36). 

Teacher-initiated interaction (column ^41) dominated the DIE, The DIE was taught 

in a whole class grouping (column 46) where the students were receiving (colxunn 
» 

52) and giving simple responses (column 54) at the cognitive level of knowledge 
(column 62). J . 

In this example, you will notice that two student activities were coded 
for one DIE: receiving and simple responses. In order to allow the natural 
setting to be accurately reflected in our analysis, we coded any dimensions that 
occurred, regardless of .the number of entries for that component. For cognitive 
level, however, we only allowed one entry per iTlE. This is so because we coded 
the most complex dimerision 'reached during any DIE for the cognitive level 
component. * . ' . ' 

Finally, upon the completion of the coding" of all lessons, the data were 

•conlbined into tables like Exhibit A based upon sinriilarity of treatment. This last 

/ 

procedure involved two steps. Given an Indication of presence or absence, we 
ftjst figured the proportion of each DIE accounted for by any one dimension. 
This proportion was derived, by dividing the recorded number of DIE dimensions 
by the frequency of DIE catogiyios . Thus, if the DIE category labeled presenta- 
tion of new content occurred 30 tim^5-«i..a^iven treatment, and basal texts were 
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,used three times in those DIE'S, then we could say that ten percent of the pre- 
sentation of new content DIE^^ for that treatment used basal texts. 

In a similar manner, by using the time duration.of each DIE In minutes 
of columns 25 and 26 on Exliibit B, and substituting it for all the ^^ones'^ recorded 
for that DIE entry, we arrived at a percentage that indicated the amount of time 
in minutes accounted for by any DIE category' Thus, In the example described 
before, if the 30 DIE'S totaled 900 minutes and the three uses of basal texts 
totaled 100 minutes, tKen we could say that 11 percent of the presentation of new 
content DIE's for that treatment used basal texts. Referring to Exhibit A, you 
Avill notice that we used' percentage of time ifi the analysis of all components 
except cognitive level where we used percentage of the WE's. 

VI. So'Me Findings 

We turn riow to a brief summary of some of our findings in order to 

illustrate the utility of our analysis system.* 

One of the potential contributions of our analysis system to knowledge 

of the instructional process is the finding that assignments DIE'S had a different 
t 1 * . ^ 

charac^r than the other three DIE categories. This can be readily seen ia the 

r ^ ' ' . ^ 

analysis of social studies instruction, but the concept also was applicable to the 
mathematics and communication^ analyses. r ' . 

i 

» 

Assignments DIE'S, the more student-oriented DIE'S, represented 32 
percent of 'the instruction in the 309 social studies lessons analyzed. Assignments, 

which were generally required of the whole class, tended to be, completed alone \ 

^ ' - • » 

(G2 percent) or in small groups (37 percent) utilizing student/student (37 percent) 

••- . ^ 18 
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or no (25 percent) interactioh patterns or an interaction patteni where tlie teachers 
/" ^ • . . 

conversed with individual students (50 percent)* as they worked. Asffimment DIE's 
tended to de-cmphasize Icnowledge goals (52 percent of the DIE^s) in favor of 

T 

comprehension (IG percent of the DIE^s) and the otlier more complex cognitive 

I 

levels (31 percent of the DEE's), 

For tlie Other £hree DIE^s, 68 percent of tlie social studies -instruction 
observed, classes were taught as whole groups -(87 percent) using teacher-Initiated 
interaction patterns (57 percent) with Icnowledge (7G percent of the DIE's) as the 
primary cognitive outcome. 

<^ 

In addition, the major findings concerning $ome of the's'ocial studies 

1. Two nationally-developed curriculum packages by the same devel- 
opeTs, Man: A Course of Study (MA COS) for fifth grade students ^n^^ People and 
Technology (PAT) for, sixUi graders, were foimd to facilitate the more comple^:^ 
cognitive levels more consibten.tly than any Other social studies curricula used in 
the district under study (respectively, 44 and 44 percent of the DIE^s were coded 
at or above the comprehension level). 

2. Primary teachers tended to develop their own social studies curric- 

> 

ular and instructional materials rather than to follow tlie plan of organi:?ation of 
published textbooks or curriculum packages. One consequence of this approach 
v^as a heavy emphasis on laiowledge goals {32 percent of the DIE's). . ■ " * 

* 

3. Senior high -school social studies teachers, even though they used 
a sizable amount of teacher-developed materials (4p percent of iixe time), were 
able to move above laiowlcdgc objectives to approximately the same proportions 
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• I* 

MA COS and PAT (47 percent of the DIE'S were t?oded at or above the comprc- 
hens ion level), * •* . , ^ ^ 

VIL Conclusion 

The establishment of trends from a large amount of qualitative tlata is? 

an extremely cumbersome and difficult task. The specific use of our analysis ' 

scheme is to enable rese'archers or e"^lu^tors to reduce a mass of qualitat|y^ 

data into a manageable form (categorical data) so that instruction can^ de'scribed 

/ . ^ 

more precisely than when one aliemptg to^proceed directly from (jualitative, data. 
While the staple is set for more systematic quantitative analyses, the richness of \ 
qualitative data for indepth description and analysis is also maintained • The 
analysis system we have described enables us to convert the outputs of qiialitar 
tive analysis to inputs (independent variables) for the more traditional quantitative ^ 
model? which »test the effect of treatments on outcomes. 
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APPENDIX: INTER-RATER RELIABILITY 



Three samples of inter'-a'atcr reliabilitj^ were eondueled to aseertain the 
three observers^ ability to use our analysis system to eode their handwritten 
trans^eripts in a. similar fashion. Eaeh observer eoded tlie same lesson transcript, 
Th(5ffi the pcreent of times eaeh iniv of observers tallied the units identieally was 
computed using the formula: ^ 

number of units of data tallied ident ically 
pefe^nt agreement = ^^..^^ number of units of data in reliability sample 

'Eaeh sample was required to have at least 100 units of data. 



The first two reliability samples were eondueted in a span of about four 
days near the beginning of the coding process. The first sample used tAvo type- 
written, verbatim, elementary social sb-idies transcripts coilected by one of the 
observers several years ago. Wnen the first sample was taken each observer 
had coded three of his or her own traiiscripts^ After extensive discussion, each 

' observer then rechceked the three original transcripts and continued on. The 
second sample used a handwritten elementary mathematics transcript taken in the 

^"^^strict as part of the mathematics sample. The tliird sample utilized a hand- 
written secondary communications transcript also collected as part of the commu- 
ricp.tion? sam.ple fn the district. By the tiine the third sample v/as taken, the 
observers had coded 41, 49, and 32 percent, respectively, of his^ or her own 
t!fanscripts. The table on this page illustrates the percent agreement of the 
three coders. 



PERCENT AGREEMENT OF THREE CODERS USING ^ 
A SYSTEM FOR CODIFYING HA ND\VRITTEN' LESSON 
TRANSCRIPTS FOR COMPREHENSIVE ANALYSIS 



Transcript 


Subject 


#2 


#2, #3 


#1, #3 


Average 


1 


ss- 


88. 5 


86.2 


81. 0 


85.2 


2 


M 


9i. 8 


91-.0 


87.7 


90.2 • 


3 


- C 


92. 3 


95.4 


< 93.8 


93.8 


Average 




90. 9 


90.9 


8 7. "5 


89.7 



■'"D. J. Fox, The Roscirch Process in Education , New York, Holt, 
Rinchart and Win.ston, 19G9, pp. 3G5-G7. 
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